Add EndNote XML Exporter + Rehaul Importer #11157

subhramit · 2024-04-07T09:47:07Z

Closes #11137
I am attempting to write an EndNote XML Exporter for JabRef.
I have successfully been able to implement the exporter, but there are a few catches, which will hopefully be addressed in follow-up issues and PRs:

~~Journal/Booktitle values beginning with "IEEE" are not captured:~~
Original Library:

On export and import:
~~The author/editor of the third entry changes from "Madrigal" to "Marigal and o S." (refer to the same pair of screenshots).~~
@koppor @Siedlerchr @HoussemNasri @ThiloteE @calixtus ~~The code is in a very raw stage right now~~, if you have time to go through it and suggest enhancements, please do.

Importer changes:
The importer had to be tweaked a little to include fields such as tertiary-title corresponding to the Book title/Journal, otherwise the column would remain empty:

Similarly, many other fields that were not retrieved by the importer as visible in BibTeX source were added to create a one-to-one mapping between the source entry and the one produced by a round trip export-import. The function convertRefNameToType was also modified to add some Entrytypes that were not recognized before.

Update: Major changes have taken place, please follow the discussion for details.

Mandatory checks

Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

…issue-11135

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java

src/main/java/org/jabref/logic/exporter/ExporterFactory.java

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java

…issue-11137

…bref into fix-for-issue-11137

Siedlerchr · 2024-04-08T18:33:40Z

In general looks good to me. As a follow up task and PR you can think about converting it to a StreamWriter (StaX).
StaX is more memory efficient. See e..g ModsExporter as an example

subhramit · 2024-04-08T20:20:21Z

In general looks good to me. As a follow up task and PR you can think about converting it to a StreamWriter (StaX). StaX is more memory efficient. See e..g ModsExporter as an example

Yes, in fact I had a couple of plans for the follow up PRs I'm going to put regarding this.

Fixing the IEEE booktitle (point 1 of PR description) (might raise this as an issue once this is merged)
Writing a test for this
StaX as you suggested - will read up on this.

subhramit · 2024-04-08T21:23:08Z

@Siedlerchr On resolving the changelog conflict on Github itself, it's saying I "dismissed" your review. I'm sorry, that was accidental.

Siedlerchr · 2024-04-08T22:23:53Z

Please do add the test here well. This makes the migration to stax easier Because you then have already a working test to verfiy the functionality.

subhramit · 2024-04-08T22:28:23Z

Please do add the test here well. This makes the migration to stax easier Because you then have already a working test to verfiy the functionality.

On it.

subhramit · 2024-04-17T12:01:58Z

Export & Import for alt-title added and tests adapted accordingly.

koppor · 2024-04-17T12:18:39Z

.../resources/org/jabref/logic/importer/fileformat/EndnoteXmlImporterTestLabelAndMultiTitle.bib

@@ -1,5 +1,6 @@
 @article{,
      address = {Cent States Forest Expt Stn, Columbus, OH USA},
+      alt-title = {Ecology},


improvement: if alt-title matches journal, then remove alt-title. Maybe, a cleanup at the end.

Should the same apply if booktitle matches alt-title?

Yes, I have done that.

koppor · 2024-04-17T12:23:42Z

.../resources/org/jabref/logic/importer/fileformat/EndnoteXmlImporterTestLabelAndMultiTitle.bib

@@ -15,6 +16,7 @@ @article{
 }
 @book{,
      address = {Univ Chicago, Chicago, IL USA},
+      alt-title = {Ecology},


(comment for below)

Here, I see journal set. This is wrong. If the entry is not an article, that field should go into booktitle or another field. Please try to check the table https://en.wikibooks.org/wiki/LaTeX/Bibliography_Management#BibTeX and think on a case-by-case basis. As fallback, just use booktitle before loosing the field.

In the original import test xml, it was present as alt-title.
Somewhere I had read during all this that booktitle in the bib is handled by secondary-title in the Endnote xml format. So I'll modify the bib and the xml, and change the StandardField of secondary-type to BOOKTITLE.

Nope, have to undo this. It's importing as booktitle even for articles. Will try to do it the way you suggested.

koppor · 2024-04-17T18:35:15Z

Review hint: The diff between two commits can be retrieved by URL hacking: 60f3ae2...0527ec7

koppor

Minor comments

koppor · 2024-04-17T18:35:20Z

src/main/java/org/jabref/logic/importer/fileformat/EndnoteXmlImporter.java

                    }
-                    case "urls" -> {
-                        handleUrlList(reader, fields, linkedFiles);
+                    case "periodical", "alt-periodical" -> {


With this, the order periodical and alt-periodical matters. Just leave out alt-periodical - or store it in the field alt-periodical (and keep it if it differs from periodical).

Checking every sub-field of alt-periodical felt tedious, and also on checking the Endnote dtd I see no alt-periodical. So I decided to leave out.
When I leave out alt-periodical, a journal field comes up in case of a Book. I might be missing something very obvious. Let us take a look at a test case:

It is as if it is ignoring the parent being unrecognized alt-periodical, and going in and setting the full-title into the journal field.
How do I prevent this?

Ah, found it! It was getting imported in the default case. Fixed it.

koppor · 2024-04-17T18:36:56Z

src/main/java/org/jabref/logic/importer/fileformat/EndnoteXmlImporter.java

-    }
+        // Cleanup: Remove alt-title if it matches the journal
+        String journalOrBooktitle = entry.getField(StandardField.JOURNAL).or(() -> entry.getField(StandardField.BOOKTITLE)).orElse("");
+        if (entry.hasField(new UnknownField("alt-title"))) {


exctract new UnknownField("alt-title") to class constant FIELD_ALT_TITLE. You can use intellij refactoring extract field - Ctrl+Shift+F.

subhramit added 4 commits April 5, 2024 17:54

Fix LICENSE link

1d3a3ac

Add Endnote XML Exporter

316753f

Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…

eed6822

…issue-11135

Fix Exporter and Importer - stage 1

5da187a

Siedlerchr reviewed Apr 7, 2024

View reviewed changes

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java Outdated Show resolved Hide resolved

Siedlerchr reviewed Apr 7, 2024

View reviewed changes

src/main/java/org/jabref/logic/exporter/ExporterFactory.java Outdated Show resolved Hide resolved

Siedlerchr reviewed Apr 7, 2024

View reviewed changes

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java Outdated Show resolved Hide resolved

subhramit changed the title ~~Fix for issue 11137~~ Add Endnote XML Exporter Apr 7, 2024

Review changes-1

1b211e8

subhramit requested a review from Siedlerchr April 7, 2024 13:36

subhramit added 2 commits April 7, 2024 23:59

Changelog addition

c84619e

Changelog line location change

06a314e

HoussemNasri reviewed Apr 7, 2024

View reviewed changes

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java Outdated Show resolved Hide resolved

HoussemNasri reviewed Apr 7, 2024

View reviewed changes

src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java Outdated Show resolved Hide resolved

subhramit and others added 4 commits April 8, 2024 17:20

Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…

cad0157

…issue-11137

Merge branch 'JabRef:main' into fix-for-issue-11137

a595cdd

Review changes-2 (Majorly formatted XML)

571e226

Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…

ef7a69b

…bref into fix-for-issue-11137

subhramit requested a review from HoussemNasri April 8, 2024 11:55

subhramit marked this pull request as ready for review April 8, 2024 11:56

Siedlerchr previously approved these changes Apr 8, 2024

View reviewed changes

Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Apr 8, 2024

Merge branch 'main' into fix-for-issue-11137

7108400

subhramit dismissed Siedlerchr’s stale review via 7108400 April 8, 2024 21:03

subhramit requested a review from Siedlerchr April 8, 2024 21:42

subhramit and others added 4 commits April 17, 2024 17:15

Adapt Importer test bib's for alt-title

a3bb94d

Add export for alt-title

7d9763d

Removed TODO comment (complete)

fdb465d

Merge branch 'main' into fix-for-issue-11137

96b34bc

koppor reviewed Apr 17, 2024

View reviewed changes

subhramit added 7 commits April 17, 2024 18:47

Cleanup operation if journal matches alt-title

eba7afd

Update expected bib wrt cleanup

60f3ae2

Update expected xml titles to include secondary-titles

07bf22c

Handle case of Books/Non-article journal -> booktitle

ca8e5f7

Journal (alt-periodical) check for book

08393ba

Add cleanup for booktitle==alt-title

6af33a3

More efficient cleanup

0527ec7

subhramit requested a review from koppor April 17, 2024 17:07

koppor requested changes Apr 17, 2024

View reviewed changes

Review changes - minor comments

794376f

subhramit requested a review from koppor April 17, 2024 19:07

koppor approved these changes Apr 17, 2024

View reviewed changes

koppor added this pull request to the merge queue Apr 17, 2024

Merged via the queue into JabRef:main with commit 7063d2d Apr 17, 2024
20 checks passed

subhramit deleted the fix-for-issue-11137 branch April 20, 2024 19:05

subhramit added type: enhancement type: feature import export / save and removed status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers labels Aug 16, 2024

subhramit mentioned this pull request Aug 22, 2024

Upgrade EndNote XML exporter to StAX #11665

Closed

6 tasks

subhramit changed the title ~~Add Endnote XML Exporter + Rehaul Importer~~ Add EndNote XML Exporter + Rehaul Importer Aug 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add EndNote XML Exporter + Rehaul Importer #11157

Add EndNote XML Exporter + Rehaul Importer #11157

subhramit commented Apr 7, 2024 •

edited

Loading

Siedlerchr commented Apr 8, 2024

subhramit commented Apr 8, 2024

subhramit commented Apr 8, 2024 •

edited

Loading

Siedlerchr commented Apr 8, 2024

subhramit commented Apr 8, 2024

subhramit commented Apr 17, 2024

koppor Apr 17, 2024

subhramit Apr 17, 2024

koppor Apr 17, 2024

subhramit Apr 17, 2024

koppor Apr 17, 2024

subhramit Apr 17, 2024

subhramit Apr 17, 2024

koppor commented Apr 17, 2024

koppor left a comment

koppor Apr 17, 2024

subhramit Apr 17, 2024 •

edited

Loading

subhramit Apr 17, 2024

koppor Apr 17, 2024

subhramit Apr 17, 2024

Add EndNote XML Exporter + Rehaul Importer #11157

Add EndNote XML Exporter + Rehaul Importer #11157

Conversation

subhramit commented Apr 7, 2024 • edited Loading

Mandatory checks

Siedlerchr commented Apr 8, 2024

subhramit commented Apr 8, 2024

subhramit commented Apr 8, 2024 • edited Loading

Siedlerchr commented Apr 8, 2024

subhramit commented Apr 8, 2024

subhramit commented Apr 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koppor commented Apr 17, 2024

koppor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subhramit Apr 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subhramit commented Apr 7, 2024 •

edited

Loading

subhramit commented Apr 8, 2024 •

edited

Loading

subhramit Apr 17, 2024 •

edited

Loading